Conversation
Remove packages with zero imports across the entire codebase: environment.yml: - PyGithub: no 'import github' found anywhere - tqdm: no direct imports (transitive dep of transformers) - mistletoe: no imports (markdown parser, unused — docs use docusaurus/sphinx) - black==22.3.0: duplicate of black[jupyter]==22.3.0 (which is a superset) build.sbt: - httpclient5: zero imports of org.apache.hc.client5 (code uses v4 API) - spark-tags: zero imports of org.apache.spark.tags (exclusion rule kept) - httpmime: moved to test scope (only imported in FabricOperations test) - httpclient: added explicitly — was previously a silent transitive dep of httpmime, needed in compile scope by RESTUtils, HTTPSchema, etc. Validated: sbt compile, Test/compile, scalastyle all pass.
|
Hey @BrendanWalsh 👋! We use semantic commit messages to streamline the release process. Examples of commit messages with semantic prefixes:
To test your commit locally, please follow our guild on building from source. |
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. Scanned FilesNone |
There was a problem hiding this comment.
Pull request overview
This PR trims unused build and environment dependencies to reduce maintenance overhead and avoid carrying packages that aren’t referenced in SynapseML’s Scala/Python codepaths.
Changes:
- Remove unused Python pip packages from
environment.yml(PyGithub, tqdm, mistletoe, and the redundantblack==22.3.0pin). - Remove unused Scala/Spark deps from
build.sbt(spark-tags,httpclient5) and adjust Apache HttpComponents scoping (addhttpclientto compile; movehttpmimeto test).
Show a summary per file
| File | Description |
|---|---|
| environment.yml | Removes pip packages with zero imports and deduplicates black pinning via black[jupyter]. |
| build.sbt | Drops unused deps, removes HttpClient 5, explicitly adds HttpClient 4 to compile scope, and scopes httpmime to tests. |
Copilot's findings
- Files reviewed: 1/2 changed files
- Comments generated: 1
| - nbconvert | ||
| - nbformat | ||
| - pyyaml | ||
| - PyGithub | ||
| - tqdm | ||
| - ipython | ||
| - pytest-codeblocks |
There was a problem hiding this comment.
PR description says tqdm remains available transitively via transformers v4.67.3, but environment.yml pins transformers==4.49.0. Please update the PR description (or adjust the validation note) to match the actual pinned version so readers can trust the dependency rationale.
Summary
Remove packages with zero imports across the entire codebase.
environment.yml
import githubfound anywhere in the repotransformers(v4.67.3)black[jupyter]==22.3.0(superset)build.sbt
org.apache.hc.client5— codebase uses v4 API (org.apache.http)org.apache.spark.tags(exclusion rule kept to prevent transitive pull)% "test"scope — only imported inFabricOperationstestRESTUtils,HTTPSchema,HTTPClients,ClusterUtilValidation (CI-mirror container: ubuntu:22.04, conda 24.11.1, JDK 11)
conda env createsbt compilesbt Test/compilesbt scalastyle; Test/scalastyleblack --checkmistletoeremovedPyGithubremovedtqdmstill works transitively